1,427 research outputs found
A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guarantees
Sparse high dimensional graphical model selection is a topic of much interest
in modern day statistics. A popular approach is to apply l1-penalties to either
(1) parametric likelihoods, or, (2) regularized regression/pseudo-likelihoods,
with the latter having the distinct advantage that they do not explicitly
assume Gaussianity. As none of the popular methods proposed for solving
pseudo-likelihood based objective functions have provable convergence
guarantees, it is not clear if corresponding estimators exist or are even
computable, or if they actually yield correct partial correlation graphs. This
paper proposes a new pseudo-likelihood based graphical model selection method
that aims to overcome some of the shortcomings of current methods, but at the
same time retain all their respective strengths. In particular, we introduce a
novel framework that leads to a convex formulation of the partial covariance
regression graph problem, resulting in an objective function comprised of
quadratic forms. The objective is then optimized via a coordinate-wise
approach. The specific functional form of the objective function facilitates
rigorous convergence analysis leading to convergence guarantees; an important
property that cannot be established using standard results, when the dimension
is larger than the sample size, as is often the case in high dimensional
applications. These convergence guarantees ensure that estimators are
well-defined under very general conditions, and are always computable. In
addition, the approach yields estimators that have good large sample properties
and also respect symmetry. Furthermore, application to simulated/real data,
timing comparisons and numerical convergence is demonstrated. We also present a
novel unifying framework that places all graphical pseudo-likelihood methods as
special cases of a more general formulation, leading to important insights
Generalized Pseudolikelihood Methods for Inverse Covariance Estimation
We introduce PseudoNet, a new pseudolikelihood-based estimator of the inverse
covariance matrix, that has a number of useful statistical and computational
properties. We show, through detailed experiments with synthetic and also
real-world finance as well as wind power data, that PseudoNet outperforms
related methods in terms of estimation error and support recovery, making it
well-suited for use in a downstream application, where obtaining low estimation
error can be important. We also show, under regularity conditions, that
PseudoNet is consistent. Our proof assumes the existence of accurate estimates
of the diagonal entries of the underlying inverse covariance matrix; we
additionally provide a two-step method to obtain these estimates, even in a
high-dimensional setting, going beyond the proofs for related methods. Unlike
other pseudolikelihood-based methods, we also show that PseudoNet does not
saturate, i.e., in high dimensions, there is no hard limit on the number of
nonzero entries in the PseudoNet estimate. We present a fast algorithm as well
as screening rules that make computing the PseudoNet estimate over a range of
tuning parameters tractable
Learning Gaussian Graphical Models with Latent Confounders
Gaussian Graphical models (GGM) are widely used to estimate the network
structures in many applications ranging from biology to finance. In practice,
data is often corrupted by latent confounders which biases inference of the
underlying true graphical structure. In this paper, we compare and contrast two
strategies for inference in graphical models with latent confounders: Gaussian
graphical models with latent variables (LVGGM) and PCA-based removal of
confounding (PCA+GGM). While these two approaches have similar goals, they are
motivated by different assumptions about confounding. In this paper, we explore
the connection between these two approaches and propose a new method, which
combines the strengths of these two approaches. We prove the consistency and
convergence rate for the PCA-based method and use these results to provide
guidance about when to use each method. We demonstrate the effectiveness of our
methodology using both simulations and in two real-world applications
Partial Separability and Functional Graphical Models for Multivariate Gaussian Processes
The covariance structure of multivariate functional data can be highly
complex, especially if the multivariate dimension is large, making extension of
statistical methods for standard multivariate data to the functional data
setting quite challenging. For example, Gaussian graphical models have recently
been extended to the setting of multivariate functional data by applying
multivariate methods to the coefficients of truncated basis expansions.
However, a key difficulty compared to multivariate data is that the covariance
operator is compact, and thus not invertible. The methodology in this paper
addresses the general problem of covariance modeling for multivariate
functional data, and functional Gaussian graphical models in particular. As a
first step, a new notion of separability for multivariate functional data is
proposed, termed partial separability, leading to a novel Karhunen-Lo\`eve-type
expansion for such data. Next, the partial separability structure is shown to
be particularly useful in order to provide a well-defined Gaussian graphical
model that can be identified with a sequence of finite-dimensional graphical
models, each of fixed dimension. This motivates a simple and efficient
estimation procedure through application of the joint graphical lasso.
Empirical performance of the method for graphical model estimation is assessed
through simulation and analysis of functional brain connectivity during a motor
task.Comment: 39 pages, 5 figure
Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation
Across a variety of scientific disciplines, sparse inverse covariance
estimation is a popular tool for capturing the underlying dependency
relationships in multivariate data. Unfortunately, most estimators are not
scalable enough to handle the sizes of modern high-dimensional data sets (often
on the order of terabytes), and assume Gaussian samples. To address these
deficiencies, we introduce HP-CONCORD, a highly scalable optimization method
for estimating a sparse inverse covariance matrix based on a regularized
pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal
gradient method uses a novel communication-avoiding linear algebra algorithm
and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving
parallel scalability on problems with up to ~819 billion parameters (1.28
million dimensions); even on a single node, HP-CONCORD demonstrates
scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to
estimate the underlying dependency structure of the brain from fMRI data, and
use the result to identify functional regions automatically. The results show
good agreement with a clustering from the neuroscience literature.Comment: Main paper: 15 pages, appendix: 24 page
- …